Vehicle engine sound control system and control method based on driver propensity using artificial intelligence

ABSTRACT

A vehicle engine sound control system identifies a vehicle driver by a driver smartphone or a driver biometric information detecting sensor and analyzes the music to which the identified driver listens with the driver smartphone or a vehicle infotainment system. A traveling pattern of the driver is analyze by applying any one among a vehicle, a GPS, a road, and weather as a condition. A driver propensity engine sound pattern is generated as a result value by learning at least any one information among a driver identifying unit, a music analyzing unit, and a travel analyzing unit. The engine sound is adjusted and output based the result value.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to Korean Patent Application No.10-2019-0043753, filed on Apr. 15, 2019, which is incorporated herein byreference in its entirety.

BACKGROUND Field of the Disclosure

The present disclosure relates to a vehicle engine sound control systemand control method based on driver propensity using artificialintelligence, and more particularly, to a vehicle engine sound controlsystem and control method based on driver propensity using artificialintelligence for customized engine sound control based the musical tasteand mood state of a driver.

Description of Related Art

Today, a vehicle is in a downsizing trend in response to stringentenvironmental regulations. Therefore, the number of cylinders or thedisplacement volume of an engine is also being downsized. Meanwhile, thedownsizing of the engine may be beneficial in terms of environment, butless beneficial in terms of engine sound. In recent years, vehicleconsumers tend to emphasize driving sensibility. Therefore, it is atrend to provide a driving selector switch for each of various vehiclemakers.

For example, an eco-mode is provided that emphasizes fuel economy and asports mode or a dynamic mode is provided for speed or punchy driving. Adriver may select a sport mode or a dynamic mode to enjoy dynamictraveling. Nevertheless, the sound of the downsized engine sound may bedisappointing. In addition, if the engine sound is heard the same evenif the driving mode is changed, the driver may be less satisfied.

Accordingly, functions for outputting virtual engine sound have recentlybeen developed. For example, the engine sound stored in a main board isoutput through an indoor speaker. The intensity of the sound pressure tobe output is adjusted based on the degree of engagement of anaccelerator pedal by the driver.

However, as described above, the simple adjustment of the sound pressurein proportion to the opening amount of the pedal does not improve theenjoyment of driving since the driver recognizes the sound as artificialengine sound. In addition, in the related art, noise has been eliminatedor decreased by providing a sound wave having the wavelength opposite tothe engine noise. This is also referred to as active noise cancellingtechnology.

However, if the engine sound is canceled using the active noisecancelling technique as described above, the dynamic driving sensibilityis unable to be felt. In addition, the above-described conventionaltechniques have been applied uniformly regardless of the driver. Inother words, only the predetermined virtual engine sounds for thetraveling mode selected by the driver are heard except for the currentemotional state of the driver. Therefore, there has been a limitation inthat the related art is unable to provide the engine sound thatcorresponds to the current emotional state of the driver currentlywithin the vehicle.

The contents described in this section are merely to help theunderstanding of the background of the present disclosure, and mayinclude what is not previously known to those skilled in the art towhich the present disclosure pertains.

SUMMARY

In order to overcome the above problems, an object of the presentdisclosure is to provide a vehicle engine sound control system andcontrol method based on driver propensity using artificial intelligence,which may automatically adjust engine sound that corresponds to thecurrent emotional state of a driver currently boarding a vehicle.

The present disclosure provides a vehicle engine sound control systembased on driver propensity using artificial intelligence that mayinclude a driver identifying unit configured to detect a vehicle driverby a driver smartphone or a driver biometric information detectingsensor; a music analyzing unit configured to analyze the music to whichthe detected driver listens with the driver smartphone or a vehicleinfotainment system; a travel analyzing unit configured to analyze atraveling pattern of the driver by applying any one among a vehicle, aglobal positioning system (GPS), a road, and weather as a condition; anartificial intelligence learning unit configured to generate a driverpropensity engine sound pattern as a result value by learning at leastany one information among the driver identifying unit, the musicanalyzing unit, and the travel analyzing unit; an engine soundcontroller for controlling the engine sound of the vehicle based on theresult value of the artificial intelligence learning unit; and an outputunit configured to output the adjusted engine sound.

In addition, the driver identifying unit interlocks with a near-fieldwireless communication, and may be configured to confirm driveridentification (ID) information with the driver smartphone, and confirma driver biometric recognition information with the driver biometricinformation detecting sensor, and the driver biometric informationdetecting sensor may be a driver wearable device installed within avehicle to detect the driver biometric information with a directmeasurement signal or connected with a vehicle infotainment system todetect the information with an indirect measurement signal. The musicanalyzing unit may be configured to reflect the characteristics of thetype of music or Bits of music to which the driver listens while thevehicle is being driven.

The travel analyzing unit may further include at least any one of thetraveling regional information or the traveling temporal information ofthe vehicle, and the artificial intelligence learning unit may beconfigured to learn any one or more among the age, gender, and the genreand Bits information of favorite music of the driver. In addition, theengine sound controller may be configured to output a primary enginesound based on the result learned in the artificial intelligencelearning unit, and implement a target engine sound desired by the driverby re-learning a feedback result value at which the driver has respondedto the output primary engine sound in the artificial intelligencelearning unit.

The artificial intelligence learning unit is a guidance learning methodusing recurrent neural network (RNN) and deep neural network (DNN)methods. In addition, the feedback result value reflects at least anyone changed value of the music analyzing unit and the travel analyzingunit. The engine sound controller may be configured to adjust thearrangement and level of an engine order of the vehicle. The enginesound may be any one among Powerful sound, Pleasant sound, Dynamicsound, and Sporty sound. In addition, the engine sound controller may beconfigured to adjust frequency weight information of the engine sound,and adjust equalizer (EQ) information of the engine sound. The outputunit may be at least any one among an indoor speaker of the vehicle, aresonator, and a frequency filter.

According to another aspect, the present disclosure provides a vehicleengine sound control method based on driver propensity using artificialintelligence that may include outputting primary engine sound of avehicle by learning first driving-related information regardingartificial intelligence learning information of a driver in theartificial intelligence of the vehicle; and outputting favorite targetengine sound of the driver by additionally learning seconddriving-related information generated by the artificial intelligencelearning information by the driver's response to the output primaryengine sound.

The first driving-related information may be driver ID informationconfirmed by a driver smartphone in interlock with near-field wirelesscommunication or driver biometric recognition information confirmed by adriver biometric information detecting sensor. In addition, the firstdriving-related information may be at least any one of the sound sourceinformation stored in the driver smartphone, and the characteristicsinformation of the type of music or Bits of music to which the driverlistens while the vehicle is being driven. The first driving-relatedinformation may include at least any one among an acceleration pattern,a shift pattern, a brake pattern, and a fastest use pattern

Further, the first driving-related information may include at least anyone of the traveling regional information or the traveling temporalinformation of the vehicle. The artificial intelligence may use aguidance learning method using RNN and DNN methods. In addition, thesecond driving-related information may be frequency weight informationof the engine sound or equalizer EQ information of the engine sound. Thetarget engine sound may be adjusted by at least any one among avibration-based engine sound control (ESEV), a virtual sound sourcecontrol (ASD), and structure-borne noise control (ESG). The targetengine sound may be output through at least any one among an indoorspeaker of the vehicle, a resonator, and a frequency filter.

According to the present disclosure as described above, the followingeffects can be obtained.

Firstly, it may be classified into various types by reflecting thetraveling style analysis of the vehicle, the types of music to belistened to while driving, and furthermore, the characteristics of theBits of music, and thus, it may be possible to perform the additionalcontrol based the characteristics of the driver in the engine soundcontrol algorithm such as the engine vibration-based engine soundcontrol technology or the virtual sound source control technology,thereby automatically providing the individual engine sound thatsatisfies the sensibility of the driver.

Secondly, it may be possible to automatically detect the driver withinthe vehicle and perform the engine sound control by the artificialintelligence using the musical taste information of the correspondingdriver, thereby maximizing the enjoyment of driving since the enginesound is output without operation of the driver.

Thirdly, it may be possible to perform the feedback control for thedriving propensity or the musical propensity of the driver in real time,thereby providing the engine sound optimized for the current emotionalstate of the driver to maximize the driver satisfaction.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features of the present invention will now bedescribed in detail with reference to certain exemplary embodimentsthereof illustrated in the accompanying drawings which are givenhereinbelow by way of illustration only, and thus are not limitative ofthe present invention, and wherein:

FIG. 1 is a diagram showing an overall configuration of a systemaccording to an exemplary embodiment of the present disclosure;

FIG. 2 is a diagram showing a procedure of an artificial intelligencelearning unit according to an exemplary embodiment of the presentdisclosure to confirm the traveling propensity of the driver;

FIG. 3 is a diagram showing a feedback procedure according to anexemplary embodiment of the present disclosure;

FIG. 4 is a diagram showing a configuration example of a driverbiometric information recognizing device according to an exemplaryembodiment of the present disclosure;

FIG. 5 is a flowchart of the driver biometric information usingartificial intelligence according to an exemplary embodiment of thepresent disclosure;

FIGS. 6 and 7 are flowcharts of a vibration-based engine sound control(ESEV) according to an exemplary embodiment of the present disclosure;and

FIGS. 8 and 9 are block diagrams showing a vehicle engine sound controlmethod based on the driver propensity and the driver biometricinformation using artificial intelligence according to an exemplaryembodiment of the present disclosure.

DETAILED DESCRIPTION

It is understood that the term “vehicle” or “vehicular” or other similarterm as used herein is inclusive of motor vehicles in general such aspassenger automobiles including sports utility vehicles (SUV), buses,trucks, various commercial vehicles, watercraft including a variety ofboats and ships, aircraft, and the like, and includes hybrid vehicles,electric vehicles, plug-in hybrid electric vehicles, hydrogen-poweredvehicles and other alternative fuel vehicles (e.g. fuels derived fromresources other than petroleum). As referred to herein, a hybrid vehicleis a vehicle that has two or more sources of power, for example bothgasoline-powered and electric-powered vehicles.

Although exemplary embodiment is described as using a plurality of unitsto perform the exemplary process, it is understood that the exemplaryprocesses may also be performed by one or plurality of modules.Additionally, it is understood that the term controller/control unitrefers to a hardware device that includes a memory and a processor. Thememory is configured to store the modules and the processor isspecifically configured to execute said modules to perform one or moreprocesses which are described further below.

Furthermore, control logic of the present disclosure may be embodied asnon-transitory computer readable media on a computer readable mediumcontaining executable program instructions executed by a processor,controller/control unit or the like. Examples of the computer readablemediums include, but are not limited to, ROM, RAM, compact disc(CD)-ROMs, magnetic tapes, floppy disks, flash drives, smart cards andoptical data storage devices. The computer readable recording medium canalso be distributed in network coupled computer systems so that thecomputer readable media is stored and executed in a distributed fashion,e.g., by a telematics server or a Controller Area Network (CAN).

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the disclosure.As used herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof. As used herein, the term “and/or”includes any and all combinations of one or more of the associatedlisted items.

Unless specifically stated or obvious from context, as used herein, theterm “about” is understood as within a range of normal tolerance in theart, for example within 2 standard deviations of the mean. “About” canbe understood as within 10%, 9%, 8%, 7%, 6%, 5%, 4%, 3%, 2%, 1%, 0.5%,0.1%, 0.05%, or 0.01% of the stated value. Unless otherwise clear fromthe context, all numerical values provided herein are modified by theterm “about.”

Various modifications and various forms may be made in the presentdisclosure, so that specific exemplary embodiments are illustrated inthe drawings and described in detail in the specification. It should beunderstood, however, that it is not intended to limit the presentdisclosure to the particular disclosed forms, but includes allmodifications, equivalents, and alternatives falling within the spritand technical scope of the present disclosure.

Like reference numerals are used for like elements in describing eachdrawing. The terms “first,” “second,” and the like can be used toillustrate various components, but the components should not be limitedby the terms. The terms are used to differentiate one element fromanother. For example, a first component can be referred to as a secondcomponent, and similarly, the second component may be also referred toas the first component without departing from the scope of the presentdisclosure.

Unless otherwise defined, all terms including technical and scientificterms used herein have the same meaning as commonly understood by one ofnormal skill in the art to which the present disclosure belongs. It willbe further understood that terms, such as those defined in commonly useddictionaries, should be additionally interpreted as having a meaningthat is consistent with their meaning in the context of the relevantart, and will not be interpreted in an idealized or overly formal senseunless expressly so defined in the application.

A vehicle engine sound control system based on driver propensity usingartificial intelligence according to an exemplary embodiment of thepresent disclosure will be described with reference to the accompanyingdrawings. FIG. 1 is a diagram showing an overall configuration of asystem according to an exemplary embodiment of the present disclosure. Avehicle engine sound control system based on driver propensity usingartificial intelligence may include a driver identifying unit 100, amusic analyzing unit 100-1, an artificial intelligence learning unit200, a travel analyzing unit 300, an engine sound controller 400, anoutput unit 500, and a driver biometric information recognizing device600. The various components of the system may be operated by a vehiclecontroller or other overall controller within the vehicle specificallyprogrammed to execute the functions of each of the units.

Particularly, the driver identifying unit 100 may be configured todetect the vehicle driver. In addition, the driver identifying unit 100may be configured to analyze the musical propensity of a driver byanalyzing the music information of the smartphone held by the driverusing the music analyzing unit 100-1 or analyze the musical propensityof the driver by detecting the driver biometric information using thedriver biometric information recognizing device 600. The music analyzingunit 100-1 may be configured to analyze the music information that thedriver enjoyed in the past and also the music that the driver iscurrently listening to in real time, and the analysis items may includethe genre, Bits, and musician information, etc. of the music.

The music analyzing unit 100-1 may be connected with the driveridentifying unit 100 to analyze the musical taste of the driverregardless of whether the driver is detected. For this purpose, themusic analyzing unit 100-1 may be connected to an indoor tone colormeasuring sensor 101. The indoor tone color measuring sensor 101 may beconfigured to detect the music information reproduced in the indoor byfrequency to provide it to the music analyzing unit 100-1.

For example, when the driver is detected in the driver identifying unit100, the music analyzing unit 100-1 may be configured to analyze themusical taste using the music information heard in the vehicle duringthe past traveling confirmed through the identified existing informationof the driver (e.g., music list or play list stored in the driversmartphone, or a vehicle infotainment system (see FIG. 4) interlockedwith the driver biometric information recognizing device 600) or themusic information generated in the vehicle infotainment system (see FIG.4) interlocked with the driver biometric information recognizing device600. On the other hand, the music analyzing unit 100-1 may be configuredto analyze the musical taste of the driver by analyzing the musicinformation heard by the driver using the vehicle infotainment system(see FIG. 4) interlocked with the driver biometric informationrecognizing device 600 during the current traveling of the vehicle evenwhen the driver is not detected in the driver identifying unit 100.

The artificial intelligence learning unit 200 may be configured to learnat least any one information of the driver identifying unit 100, themusic analyzing unit 100-1, the travel analyzing unit 300, and thedriver biometric information recognizing device 600. For example, theartificial intelligence learning unit 200 may be configured to learn thetaste, traveling propensity, or favorite emotional characteristic, etc.of the driver based on any one or more learning information of thedriver identifying unit 100, the music analyzing unit 100-1, and thetravel analyzing unit 300. In addition, the artificial intelligencelearning unit 200 may be configured to learn the taste, travelingpropensity, or favorite emotional characteristic, etc. of the driverbased on the driver biometric information of the driver biometricinformation recognizing device 600 together with any one or morelearning information of the driver identifying unit 100, the musicanalyzing unit 100-1, and the travel analyzing unit 300. Therefore, theartificial intelligence learning unit 200 may be configured to generatea driver propensity engine sound pattern as a result value.

The travel analyzing unit 300 may be configured to analyze the travelingpattern of the driver by applying any one of a vehicle, a GlobalPositioning System (GPS), a road, and weather. In particular, the travelanalyzing unit 300 may be configured to analyze the traveling conditionof the vehicle, the engine state condition, the GPS condition, thereal-time road condition, and the weather environment condition, etc.The engine sound controller 400 may be configured to adjust the enginesound of the vehicle based on the driver propensity engine sound patternthat is the result value of the artificial intelligence learning unit200.

In particular, the engine sound controller 400 may be configured toperform vibration-based engine sound control (ESEV) (see FIGS. 6 to 9),which is a preferred embodiment of the present disclosure. Furthermore,the engine sound controller 400 may be configured to perform a virtualsound source control (ASD) or a structure-borne noise control (ESG),etc. according to another exemplary embodiment of the presentdisclosure. The output unit 500 may be configured to output the adjustedengine sound. The output unit 500 may be configured to output the soundvia a vehicle indoor speaker, a resonator, or a frequency filter.

FIG. 2 is a diagram showing a procedure in which the artificialintelligence learning unit 200 according to an exemplary embodiment ofthe present disclosure confirms the traveling propensity of the driverin association with the driver identifying unit 100. The methodsdescribed herein may be executed by the overall controller discussedabove. For example, the driver identifying unit 100 may include a driverID identifying unit 110, an identification driver information unit 120,and unidentified driver information 121.

The artificial intelligence learning unit 200 may include an artificialintelligence module 201, a category module 202, and a driver travelingcharacteristic feedback module 230. In particular, the category module202 may be classified into a first graph 210 and a second graph 220. Thefirst graph 210 may be an initial generation result in which theartificial intelligence module 201 has used data, and the second graph220 may be a learning generation result in which the artificialintelligence module 201 has re-learned and processed based on the driverreaction, etc. to the first graph 210 through the feedback control 230(see FIG. 4). Therefore, the first graph 210 and the second graph 220have been the same component but classified for explanation.

Therefore, a procedure of confirming the traveling propensity of thedriver is as follows. The driver identifying unit 100 may be configuredto detect the vehicle driver by confirming the driver ID with the driverbelongings (e.g., items within the vehicle). More specifically, thedriver identifying unit 100 may use Bluetooth or Blue Link of the driversmartphone, which is a type of driver belonging connected with thevehicle, to confirm the driver ID.

In addition, the driver identifying unit 100 may use the biometricinformation to confirm the driver ID. The driver biometric informationmay be mounted on the cluster instrument panel of the vehicle, and apupil recognizing device of a window biometric sensor 603 (see FIG. 4)constituting the driver biometric information recognizing device 600 maybe used, or a fingerprint recognizing device or a voice recognizingdevice, etc. may be used.

Furthermore, the driver identifying unit 100 may be configured toconnect with the driver biometric information via Bluetooth or Blue Linkin connection with any one of a steering wheel biometric sensor 601, awindow biometric sensor 603, and a wearable biometric sensor 607 of thedriver biometric information recognizing device 600. When detecting thedriver ID, the driver identifying unit 100 may be configured to confirmthe driver database (DB) such as the driver information, the storedsound source data, or the favorite music site by the driver informationacquired from the identified driver of the identification driverinformation unit 120 acquired from the identified driver ID.

Herein, the driver information may be a taste, traveling propensity, andfavorite emotional characteristics of the driver that have beenpreviously learned, which is similar to the ID characteristic describedbelow. Conversely, when not detecting the driver ID, the driveridentifying unit 100 may be configured to confirm the music DB of thedriver using the music hearing while the vehicle is being driven, themusic becoming a DB and stored for a particular period of time, etc. bythe driver information acquired from the unidentified driver of theunidentified driver information 121.

In other words, the detection of the driver ID by the driver IDidentifying unit 110 may include, for example, a state in which thedriver has entered the vehicle with the smartphone, and at this time,the driver DB may be confirmed using the Bluetooth information, etc.interlocked between the vehicle and the smartphone. Conversely, afailure of detecting or identifying the driver ID by the driver IDidentifying unit 110 may include, for example, a state where the driverhas entered the vehicle without holding a mobile device such as asmartphone capable of communicating with the vehicle, or a new driverwho is entering the vehicle for a first time.

Therefore, when having failed to detect or identify the driver ID by thedriver ID identifying unit 110, the driver identifying unit 100 may beconfigured to confirm the musical taste of the driver from the music towhich the driver has listened to through the infotainment system whileoperating the vehicle. The artificial intelligence learning unit 200 maybe configured to perform artificial intelligence learning using thedriver musical taste information acquired from the driver information ofthe identification driver information 120 acquired from the driver IDrecognized in the artificial intelligence module 201 or the driverinformation of the unidentified driver information 121 acquired from thedriver unrecognized therein.

The artificial intelligence module 201 of the artificial intelligencelearning unit 200 may be operated in interlock or connection with themusic analyzing unit 100-1. Accordingly, the music analyzing unit 100-1may reflect the characteristics of the type of music or the Bits ofmusic to which the driver listens while the vehicle is being driven oroperated. In addition, the artificial intelligence module 201 of theartificial intelligence learning unit 200 may be configured to learn anyone or more of the age and gender of the driver, and the genre and Bitsinformation of favorite music. This is described through the first graph210 and the second graph 220 of the category module 202.

The first graph 210 shows the distribution of favorite music genre orBits based on the age/gender of the driver according to the result oflearning from the artificial intelligence module 201 of the artificialintelligence learning unit 200. A first category 211, which has beenprimarily learned, may include the driver age in the 40s, the genderbeing male, and the favorite music being strong Bits music. A secondcategory 212, which has been primarily learned, may include the driverage in the 40s, the gender being female, and the favorite music beingthe music of K-POP Bits. A third category 213, which has been primarilylearned, may include the driver age in the 40s, the gender being male,and the genre of the favorite music being the classical music. A fourthcategory 214, which has been primarily learned, may include the driverage in the 20s, the gender being male, and the favorite music beingstrong Bits music. A fifth category 215, which has been primarilylearned, may include the driver age in the 30s, the gender being female,and the favorite music being the music of K-POP Bits. However, thepresent disclosure is not limited thereto.

As described above, in the first graph 210, the artificial intelligencelearning unit 200 may be configured to obtain the characteristics of theage, gender, and favorite music, the ID characteristics, etc. of thedriver as a result of the guidance learning to obtain driver travelingcharacteristic information 123 as a result. The artificial intelligencelearning unit 200 may be configured to receive the driver travelingcharacteristic information into the driver traveling characteristicfeedback module 230 to perform the additional guidance learning by theartificial intelligence module 201 of the artificial intelligencelearning unit 200 again.

The additional guidance learning may include learning additionallyreflecting the musical taste of the driver that changes in real time tothe driver propensity according to the primarily learned category. Forexample, when a 20s male driver mainly listens to strong Bits music andrecently, mainly listens to a weak Bits ballade, this may be learnedwith artificial intelligence in the additional guidance learning to bestored as a corresponding driver DB.

The second graph 220 shows the distribution of the traveling propensityand the favorite music of the driver through the additional guidancelearning. The first category 221, which has been additionally learned,shows the traveling propensity of the driver as mainly driving in theECO mode and the listening type of music being strong Bits music, as theinformation acquired by performing the additional guidance learning forthe first category 211, which has been primarily learned.

The second category 222, which has been additionally learned, shows thetraveling propensity of the driver as mainly driving in the NORMAL modeand the listening type of music being the music of K-POP Bits, as theinformation acquired by performing the additional guidance learning forthe second category 212, which has been primarily learned. The thirdcategory 223, which has been additionally learned, shows the travelingpropensity of the driver as mainly driving in the NORMAL mode and thelistening type of music being the classical music, as the informationacquired by performing the additional guidance learning for the thirdcategory 213, which has been primarily learned.

The fourth category 224, which has been additionally learned, shows thetraveling propensity of the driver as mainly driving in the DYNAMIC modeand the listening type of music being strong Bits music, as theinformation acquired by performing the additional guidance learning forthe fourth category 214, which has been primarily learned. The fifthcategory 225, which has been additionally learned, shows the travelingpropensity of the driver as mainly driving in the DYNAMIC mode and thelistening type of music being the music of K-POP Bits, as theinformation acquired by performing the additional guidance learning forthe fifth category 215, which has been primarily learned. However, thepresent disclosure is not limited thereto. The engine sound controller400 may be configured to output the engine sound through the output unit500 using the traveling propensity and the favorite music information ofthe driver obtained from the artificial intelligence module 201 of theartificial intelligence learning unit 200.

FIG. 3 is a diagram showing a feedback procedure according to anexemplary embodiment of the present disclosure. In particular, theengine sound may be adjusted through a control of the arrangement andlevel of the engine order of the vehicle. In addition, the engine soundcontroller 400 may be configured to adjust the equalizer EQ of theengine sound. Then, the engine sound may be any one among Powerful sound511, Dynamic sound 512, Pleasant sound 513, and Sporty sound 514.Furthermore, the engine sound may include luxury sound. In this case,the Powerful sound is a strong sense of mid/low frequencycharacteristics, the Dynamic sound is a strong sense of speed along withmid/low frequency characteristics, the Pleasant sound is a strong senseof pure sound without discomfort characteristics, the Sporty sound is astrong sense of speed along with pure sound characteristics, and theLuxury sound is a strong sense of high-quality sound characteristicswhich is quiet and harmonizes with pure sound.

Therefore, the engine sound controller 400 may use the travelingpropensity and the favorite music information of the driver obtainedfrom the artificial intelligence module 201 of the artificialintelligence learning unit 200. When the traveling propensity of thedriver is mainly DYNAMIC mode, the engine sound controller 400 may beconfigured to output any one of the Powerful sound 511, the Dynamicsound 512, and the Sporty sound 514 with reference to the information ofthe travel analyzing unit 300 through the output unit 500. When thetraveling propensity of the driver is mainly the NORMAL mode or the ECOmode, the engine sound controller 400 may be configured to output thePleasant sound 513 or the Luxury sound with reference to the informationof the travel analyzing unit 300 through the output unit 500. Meanwhile,the output unit 500 may be at least any one of an indoor speaker of thevehicle, a resonator, and a frequency filter.

Hereinafter, the engine sound satisfied by the driver will be referredto as target engine sound. The driver may also feel that the enginesound output according to mood is not satisfactory unlike usual. Thispart may be solved through the feedback procedure. In other words, thefeedback procedure may include the feedback control 230 as a procedurefor obtaining the target engine sound satisfied by the driver. Thedriver may change the type of music, change the volume, or operate thetravel analyzing unit 300 in response to the engine sound heard throughthe output unit 500, which is referred to as a feedback control.

The artificial intelligence learning unit 200 may be configured toreceive the feedback control information into the driver travelingcharacteristic feedback module 230 to perform secondary guidancelearning, thereby implementing the target engine sound. In addition, thefeedback control information may reflect the changed value of at leastany one of the music analyzing unit 100-1 and the travel analyzing unit300. In other words, the engine sound controller 400 may be configuredto implement the target engine sound desired by the driver by outputtingthe primary engine sound based on the result learned by the artificialintelligence module 201 of the artificial intelligence learning unit200, and re-learning the feedback control information in response to theprimary engine sound output by the driver in the artificial intelligencelearning unit.

At this time, the artificial intelligence module 201 of the artificialintelligence learning unit 200 may be a guidance learning method usingRNN and DNN methods. In other words, the artificial intelligence module201 of the artificial intelligence learning unit 200 according to apreferred embodiment of the present disclosure is a Deep Learning methodfor applying the guidance learning, and it is preferable that the RNNmethod capable of classifying complex input variables and the DNN methodfor final characteristics extraction applying the accumulated DB dataare applied simultaneously.

For example, it may be assumed that the artificial intelligence module201 of the artificial intelligence learning unit 200 confirms that thepropensity and the musical propensity of a driver A are the fourthcategory 224, which has been additionally learned, and the engine soundof the Powerful sound 511 has been output through the output unit 500.When the driver A feels that the engine sound of the Powerful sound 511heard during traveling is not satisfactory and changes the type of musicto the classic, the secondary guidance learning may be performed by theartificial intelligence learning unit 200, and the vehicle may outputthe engine sound of the Pleasant sound 513 to the driver.

Meanwhile, it should be noted herein that the four types of enginesounds of the Powerful sound 511 to the Sporty sound 514 are provided asan example and not limited thereto. In addition, when the genre of themusic completely changes from K-POP to the classic as in the case of thedriver A, the type of engine sound may be also changed and provided.However, when the K-POP type is changed to a slow tempo song with aslightly slow Bits (e.g., from K-POP with strong Bits to ballade withweek Bits), the engine sound of the Powerful sound 511 may also beprovided, which has become softer, by adjusting the equalizer EQ of theengine sound of the Powerful sound 511 without changing the type of theengine sound.

Meanwhile, FIG. 4 is a diagram showing a configuration example of adriver biometric information recognizing device 600 according to anexemplary embodiment of the present disclosure. As shown, a vehicledriver seat system 1 is provided as a place for installing and mountingthe driver biometric information recognizing device 600. Specifically,the vehicle driver seat system 1 may include a steering wheel, a frontwindow frame, a room mirror frame, a cluster, a dashboard, and a driverseat, etc.

The driver biometric information recognizing device 600 may use a directmeasurement signal and an indirect measurement signal generated by adriver biometric information detecting sensor as a biometric signal, andmay be configured to perform a near-field wireless communication such asBluetooth or Blue Link. For example, the direct measurement signal maybe a heart rate signal, an electrocardiogram signal, a facial patternchange, and a body temperature change signal, and for this purpose, thedriver biometric information detecting sensor may use a steering wheelbiometric sensor 601, a window biometric sensor 603, and a seatbiometric sensor 605, installed within the vehicle to detect the driverbiometric information as the direct measurement signal. The steeringwheel biometric sensor 601 may be installed on the steering wheel todetect a change in the heart rate/electrocardiogram/hand temperature ofthe driver as a signal, the window biometric sensor 603 may be installedon a front window frame and a room mirror frame steering wheel to detecta change in facial surface/facial temperature of the driver as a signal,and the seat biometric sensor 605 may be installed on the driver seat todetect a change in body temperature/brain wave of the driver as asignal.

For example, the indirect measurement signal may be a driver biometricsignal, and for this purpose, the driver biometric information detectingsensor may use a wearable biometric sensor 607. The wearable biometricsensor 607 may be a wearable smart device or a vehicle infotainmentsystem configured to detect a biometric signal generated using a smartwatch, a band, or a medical monitoring device worn by the driver as adriver wearable device. In addition, the driver biometric informationrecognizing device 600 may include a biometric information analyzing map609, and the biometric information analyzing map 609 may be used todetermine the suitability for the vehicle engine sound control resultbased on the musical propensity of the driver implemented by theartificial intelligence learning unit 200. For example, the biometricinformation analyzing map 609 involves the collected Big data for thechange in the biometric signal in parameters, characteristiccorrelation, target engine sound control signal compensation, etc.

Then, FIG. 5 is a flowchart of the driver biometric information usingartificial intelligence according to an exemplary embodiment of thepresent disclosure. As shown, the biometric information analyzing map609 of the driver biometric information recognizing device 600 may applythe heart rate signal, the electrocardiogram signal, the facial patternchange, the body temperature change signal, etc. from the steering wheelbiometric sensor 601, the window biometric sensor 603, and the seatbiometric sensor 605 as the direct measurement signal, and apply thebiometric signal, etc. generated from a smart watch, a band, and amedical monitoring device worn by the driver from the wearable biometricsensor 607 as the indirect measurement signal.

Therefore, the biometric information analyzing map 609 helps to performa suitability determining technique on the vehicle engine sound controlresult based on the musical propensity of the driver. For example, thecorrelation between the change in biometric signal and the engine soundis as follows. Firstly, the characteristic control according to thefavorite music Bits of the driver may be followed by the Powerful soundcontrol according to the strong Bits determination in the engine soundcontrol (e.g., relationship 1).

Secondly, the collection/analysis of the measured change in biometricsignal data uses the characteristics of the number of the favorite musicBits as the reference of the change in biometric signal with an increasein heart rate/an increase in temperature/facial pattern determination(e.g., excitement and enjoyment) in the Powerful sound control (e.g.,relationship 2). Thirdly, performing an additional compensation controlof the engine sound control factor according to the change in biometricsignal of the driver may be used as the biometric signal correlationmatching target (e.g., relationship 3).

Specifically, the suitability determination of the engine sound controlresult by the biometric information analyzing map 609 may set the changein biometric signal as Big data, and may be used to extract theparameters for comparing the biometric signal characteristics before theengine sound control from the Big data with the biometric signalcharacteristics after the engine sound control, to analyze thecorrelation between the characteristics of the low frequency/mediumfrequency/high frequency Bits for each genre of music and thecharacteristics of the biometric signal, to compensate for the targetengine sound control signal using the relationship between the biometricchange characteristics change data of the driver and the engine soundcontrol characteristics, etc.

As a result, the driver biometric information recognizing device 600 maybe configured to transmit the suitability determination informationregarding the engine sound control result obtained from the biometricinformation analyzing map 609 to the artificial intelligence learningunit 200, and the artificial intelligence module 201 of the artificialintelligence learning unit 200 may be configured to determine a finalengine control sound by compensating for the output value of theparameter used in the Deep Learning-based engine sound control.

Meanwhile, FIGS. 6 and 7 are flowcharts of vibration-based engine soundcontrol (ESEV) according to an exemplary embodiment of the presentdisclosure. A vibration measuring unit 311 may include engine mountinformation and single-axis accelerometer information. A FFT analyzingunit 411 may be configured to receive vibration information such as anengine mount from the vibration measuring unit 311, and frequency bandinformation from an engine main order frequency band calculating unit422 connected to an engine revolutions per minute (RPM) information 322.A main order level real-time extracting unit 412 may be configured toextract the main order level from the FFT analyzing unit 411 in realtime. The vibration-based engine sound control (ESEV) unit 413 may beconfigured to receive the information from the main order levelreal-time extracting unit 412 and the engine main order frequency bandcalculating unit 422 to amplify the engine order arrangement and levelfor determining the engine tone color to transmit the information to amaster volume level setting unit 414.

Meanwhile, an engine RPM weight providing unit 430 may be configured toreceive the engine RPM information from the engine RPM information 322to provide weight information thereto and transmit the information tothe master volume level setting unit 414. An acceleration pedal weightproviding unit 440 may be configured to transmit a value of providing aweight to pedal opening amount information 340 to the master volumelevel setting unit 414. The master volume level setting unit 414 may beconfigured to provide the master volume level information to thefrequency band weight providing unit 415. At this time, a travelingcondition for each vehicle speed determining unit 451 may be configuredto receive the speed information from vehicle speed data 350 todetermine whether it is in a constant-speed state or in an accelerationstate, and transmit the result to a speed-based tone color controller452.

Furthermore, the speed-based tone color controller 452 may be providedto the procedure in which the master volume level setting unit 414provides the master volume level information to the frequency bandweight providing unit 415. The frequency band weight providing unit 415may be configured to apply an output signal to the output unit 500. Inthe second stage, the favorite frequency band may also be emphasizedthrough a favorite frequency band equalizer control for each driverclassification. In other words, the second driving-related informationmay be related to the adjustment of the equalizer EQ information of theengine sound. The target engine sound may be output through at least anyone of an indoor speaker of the vehicle, a resonator, and a frequencyfilter.

Hereinafter, a vehicle engine sound control method based on the driverpropensity using artificial intelligence with respect to the blockdiagram of the vehicle engine sound control method according to anexemplary embodiment of the present disclosure of FIGS. 6 and 7 will bedescribed with reference to FIGS. 8 and 9 together. In particular, FIGS.8 and 9 are block diagrams showing a vehicle engine sound control methodbased on the driver propensity and based on the driver biometricinformation using artificial intelligence, and the block diagram of thevehicle engine sound control method supplementarily explains the baseengine sound control flow as described below.

Firstly, a first stage may include outputting primary engine sound of avehicle by learning first driving-related information regardingartificial intelligence learning information of a driver in theartificial intelligence of the vehicle. The first driving-relatedinformation may be Bluetooth information or biometric recognizinginformation, etc. interlocked to the driver smartphone. Meanwhile, thevehicle driver may be a person who owns the corresponding vehicle, andmay be a person who may identify the ID through the biometricrecognizing information since the driver information remains in thevehicle.

On the other hand, if the corresponding driver information is notrecorded in the corresponding vehicle at all as a new driver and the newdriver does not hold an application capable of interlocking with thevehicle such as a smartphone, the driving musical propensity of the newdriver may be confirmed using the music being listened to while thevehicle is being driven. Meanwhile, the first driving-relatedinformation may be at least any one of the sound source informationstored in the driver smartphone, and the characteristics information ofthe type of music or the Bits of music to which the driver listensduring traveling.

In addition, the first driving-related information may include at leastany one of an acceleration pattern, a shift pattern, a brake pattern,and a fastest use pattern. This may be acquired from the controller areanetwork (CAN) communication information of the vehicle, and may be thedrive mode 321, the gear stage number information 331, the engine RPMinformation 322, the pedal opening amount information 340, and thevehicle speed information 350. The drive mode 321 selected by the drivermay include the Comfort mode, the Normal mode, the Sport mode, etc. Thegear stage number information 331 may include the gear stage numberinformation selected by the driver while operating the vehicle, and thedriver's intent may be more actively used in the automatic shiftfunction and also in the manual shift function.

Further, the engine RPM information 322, the pedal opening amountinformation 340, and the vehicle speed information 350 according to thedrive mode 321 and the gear stage number information 331 may be used asthe first driving-related information. In other words, the CANcommunication of the vehicle may be used to confirm the travelingcharacteristics of the driver. Meanwhile, the indoor tone colormeasuring sensor 101 mounted within the vehicle may be configured toanalyze the control result of the engine sound and using the enginesound Big data for the additional learning. Meanwhile, the firstdriving-related information may further include traveling regionalinformation or traveling temporal information of the vehicle.

The traveling regional information of the vehicle indicates informationof the region where the vehicle is being driven, such as whether thetraveling region is in an urban, a suburban, or a highway. The travelingtemporal information indicates information regarding whether the timezone during which the vehicle is being driven is a commute time with asubstantial amount of traffic, whether it is a dawn time with minimalvehicle traffic, etc.

The artificial intelligence learning unit 200 may be configured toreceive and learn as the input values all of the driving characteristicinformation such as an acceleration pattern, a shift pattern, a brakepattern, a fastest use pattern, etc. and the music information such asthe genre, musician, and Bits information of the music to which thedriver listens (e.g., including the sound source data stored uponinterlocking with the smartphone), and the spatial-temporal travelingenvironment condition of the main driving section. Therefore, thepreference of the engine sound according to the music preference of thedriver may be learned through Deep Learning, and at this time, theguidance learning method based on the Recurrent Neural Network (RNN) andthe Deep Neural Network (DNN) may be used.

A second stage may include outputting favorite target engine sound ofthe driver by additionally learning second driving-related informationgenerated by the artificial intelligence learning information by thedriver's response to the output primary engine sound. The target enginesound may be the driver favorite engine sound that reflects thecharacteristics of the driver. This may be achieved by adjusting thearrangement and level of the engine order through the engine soundcategory control for each driver classification. In other words, thetarget engine sound may be based on the vibration-based engine soundcontrol (ESEV). Meanwhile, according to another exemplary embodiment ofthe present disclosure, it is preferable to be adjusted by at least anyone of the virtual sound source control (ASD) or the structure-bornenoise control (ESG).

What is claimed is:
 1. A vehicle engine sound control system based ondriver propensity using artificial intelligence, comprising: a driveridentifying unit configured to identify a vehicle driver using a driversmartphone or a driver biometric information detecting sensor; a musicanalyzing unit configured to analyze music to which the identifieddriver listens with the driver smartphone or a vehicle infotainmentsystem; a travel analyzing unit configured to analyze a travelingpattern of the driver by applying any one among a vehicle, a globalpositioning system (GPS), a road, and weather as a condition; anartificial intelligence learning unit configured to generate a driverpropensity engine sound pattern as a result value by learning at leastany one information determined by the driver identifying unit, the musicanalyzing unit, and the travel analyzing unit; an engine soundcontroller configured to adjust the engine sound of the vehicle based onthe result value of the artificial intelligence learning unit; and anoutput unit configured to output the adjusted engine sound.
 2. Thevehicle engine sound control system based on the driver propensity usingthe artificial intelligence of claim 1, wherein the driver identifyingunit is configured to: connect with a near-field wireless communication,and confirm driver identification (ID) information with the driversmartphone; and confirm a driver biometric recognition information withthe driver biometric information detecting sensor, wherein the driverbiometric information detecting sensor is a driver wearable deviceinstalled within the vehicle to detect the driver biometric informationwith a direct measurement signal or connected with a vehicleinfotainment system to detect the driver biometric information with anindirect measurement signal.
 3. The vehicle engine sound control systembased on the driver propensity using the artificial intelligence ofclaim 1, wherein the music analyzing unit reflects the characteristicsof the type of music or Bits of music to which the driver listens whileoperating the vehicle.
 4. The vehicle engine sound control system basedon the driver propensity using the artificial intelligence of claim 1,wherein the travel analyzing unit further includes at least any one ofthe traveling regional information or the traveling temporal informationof the vehicle, and the artificial intelligence learning unit learns anyone or more among the age, gender, and the genre and Bits information offavorite music of the driver.
 5. The vehicle engine sound control systembased on the driver propensity using the artificial intelligence ofclaim 4, wherein the engine sound controller is configured to: output aprimary engine sound based on the result learned in the artificialintelligence learning unit, and implement a target engine sound desiredby the driver by re-learning a feedback result value at which the driverhas responded to the output primary engine sound in the artificialintelligence learning unit.
 6. The vehicle engine sound control systembased on the driver propensity using the artificial intelligence ofclaim 1, wherein the artificial intelligence learning unit is a guidancelearning method using recurrent neural network (RNN) and deep neuralnetwork (DNN) methods.
 7. The vehicle engine sound control system basedon the driver propensity using the artificial intelligence of claim 5,wherein the feedback result value reflects at least any one changedvalue of the music analyzing unit and the travel analyzing unit.
 8. Thevehicle engine sound control system based on the driver propensity usingthe artificial intelligence of claim 1, wherein the engine soundcontroller is configured to adjust the arrangement and level of anengine order of the vehicle.
 9. The vehicle engine sound control systembased on the driver propensity using the artificial intelligence ofclaim 5, wherein the engine sound is any one among Powerful sound,Pleasant sound, Dynamic sound, and Sporty sound.
 10. The vehicle enginesound control system based on the driver propensity using the artificialintelligence of claim 1, wherein the engine sound controller isconfigured to adjust frequency weight information of the engine sound,and adjust equalizer information of the engine sound.
 11. The vehicleengine sound control system based on the driver propensity using theartificial intelligence of claim 1, wherein the output unit is at leastany one among the group consisting of: an indoor speaker of the vehicle,a resonator, and a frequency filter.
 12. A vehicle engine sound controlmethod based on driver propensity using artificial intelligence,comprising: outputting, by a controller, a primary engine sound of avehicle by learning first driving-related information regardingartificial intelligence learning information of a driver in theartificial intelligence of the vehicle; and outputting, by thecontroller, a favorite target engine sound of the driver by additionallylearning second driving-related information generated by the artificialintelligence learning information by a driver response to the outputprimary engine sound.
 13. The vehicle engine sound control method basedon the driver propensity using the artificial intelligence of claim 12,wherein the first driving-related information is driver identification(ID) information confirmed by a driver smartphone in communication withnear-field wireless communication or driver biometric recognitioninformation confirmed using a driver biometric information detectingsensor.
 14. The vehicle engine sound control method based on the driverpropensity using the artificial intelligence of claim 12, wherein thefirst driving-related information is at least any one of the soundsource information stored in the driver smartphone, and thecharacteristics information of the type of music or Bits of music towhich the driver listens while operating the vehicle.
 15. The vehicleengine sound control method based on the driver propensity using theartificial intelligence of claim 12, wherein the first driving-relatedinformation includes at least one of the group consisting of: anacceleration pattern, a shift pattern, a brake pattern, and a fastestuse pattern.
 16. The vehicle engine sound control method based on thedriver propensity using the artificial intelligence of claim 12, whereinthe first driving-related information further includes at least any oneof the traveling regional information or the traveling temporalinformation of the vehicle.
 17. The vehicle engine sound control methodbased on the driver propensity using the artificial intelligence ofclaim 12, wherein the artificial intelligence uses a guidance learningmethod using recurrent neural network (RNN) and deep neural network(DNN) methods.
 18. The vehicle engine sound control method based on thedriver propensity using the artificial intelligence of claim 12, whereinthe second driving-related information is frequency weight informationof the engine sound or equalizer information of the engine sound. 19.The vehicle engine sound control method based on the driver propensityusing the artificial intelligence of claim 12, wherein the target enginesound is adjusted by at least one selected from the group consisting of:a vibration-based engine sound control (ESEV), a virtual sound sourcecontrol (ASD), and structure-borne noise control (ESG).
 20. The vehicleengine sound control method based on the driver propensity using theartificial intelligence of claim 12, wherein the target engine sound isoutput through at least one from the group consisting of: an indoorspeaker of the vehicle, a resonator, and a frequency filter.